Papers with information loss
Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications? (2024.naacl-short)
Copied to clipboard
Yotam Intrator, Matan Halfon, Roman Goldenberg, Reut Tsarfaty, Matan Eyal, Ehud Rivlin, Yossi Matias, Natalia Aizenberg
| Challenge: | Existing studies have focused on pre-translation, but there is still need for it . authors say that it is not universally necessary to translate large language models . |
| Approach: | They re-evaluate the need for pre-translation in the context of PaLM2 models . authors found that PaLM2-L consistently outperforms pre-translated in 94 out of 108 languages . |
| Outcome: | The proposed model outperforms pre-translation in 94 out of 108 languages and 6 benchmarks . authors argue that pre-translated inputs can be used to improve performance . |
Preserving Workflow Reproducibility: The RePlay-DH Client as a Tool for Process Documentation (L18-1)
Copied to clipboard
| Challenge: | a tool for elicitation and management of process metadata is presented . detailed documentation of workflows is an arduous and neglected task . |
| Approach: | They propose a software tool for elicitation and management of process metadata. |
| Outcome: | The proposed tool minimizes the additional effort required for producing a sustainable workflow documentation. |
ENT-DESC: Entity Description Generation by Exploring Knowledge Graph (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for knowledge-to-text generation use RDF triples or key-value pairs to generate a natural language description. |
| Approach: | They propose a large-scale dataset to facilitate the study of KG-to-text . they propose MGCN model architecture that incorporates aggregation methods to extract the rich graph information. |
| Outcome: | The proposed model can represent the original graph information more comprehensively and integrates multiple aggregation methods to extract the rich graph information. |
Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage (2026.findings-acl)
Copied to clipboard
Junhao Hu, Fangze Li, Mingtao Xu, Feifan Meng, Shiju Zhao, Tiancheng Hu, Ting Peng, Anmin Liu, Wenrui Huang, Chenxu Liu, Ziyue Hua, Tao Xie
| Challenge: | Prior work typically decomposes inference into prefill and decode stages, with the decode stage dominating total latency. |
| Approach: | They propose an algorithm that detects threshold where information loss exceeds information gain during sparse decoding to reduce token consumption by up to 90% and a marginal accuracy degradation of less than 2%. |
| Outcome: | The proposed algorithm reduces token consumption by 90% with a marginal accuracy degradation of less than 2% across reasoning-intensive benchmarks. |
Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for continual prompt tuning are limited by the ever-growing parameter scale of modern language models (e.g., GPT-4 that may have 1.76 trillion parameters). |
| Approach: | They propose a method for continual prompt tuning that enables the lifelong learning of a pre-trained language model by adding a task-specific prompt to a queue of older tasks. |
| Outcome: | The proposed method outperforms the state-of-the-art methods substantially on continual prompt tuning benchmarks. |
Unsupervised Few-Bits Semantic Hashing with Implicit Topics Modeling (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for few-bits hashing cannot be guaranteed due to severe information loss. |
| Approach: | They propose a simple unsupervised neural generative semantic hashing method with a focus on few-bits hash. |
| Outcome: | The proposed method improves on the state-of-the-art methods in few-bits hashing. |
InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification (2024.acl-long)
Copied to clipboard
Jan Trienes, Sebastian Joseph, Jörg Schlötterer, Christin Seifert, Kyle Lo, Wei Xu, Byron Wallace, Junyi Jessy Li
| Challenge: | Text simplification aims to make technical texts more accessible to laypeople but often results in deletion of information and vagueness. |
| Approach: | They propose a framework to characterize and recover simplification-induced information loss in form of question-and-answer (QA) pairs. |
| Outcome: | The proposed framework characterizes and recovers simplification-induced information loss in form of question-and-answer (QA) pairs. |
Multi-layer Representation Fusion for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Neural machine translation systems require a number of stacked layers for deep models, but the prediction depends on the sentence representation of the top-most layer with no access to low-level representations. |
| Approach: | They propose a multi-layer representation fusion approach to fusing stacked layers to learn a better representation from the stack. |
| Outcome: | The proposed approach yields 0.92 and 0.56 BLEU points over the strong Transformer baseline on IWSLT German-English and NIST Chinese-English MT tasks respectively. |
DocAMR: Multi-Sentence AMR Representation and Evaluation (2022.naacl-main)
Copied to clipboard
Tahira Naseem, Austin Blodgett, Sadhana Kumaravel, Tim O’Gorman, Young-Suk Lee, Jeffrey Flanigan, Ramón Astudillo, Radu Florian, Salim Roukos, Nathan Schneider
| Challenge: | Abstract Meaning Representation (AMR) graphs are compared to gold graphs by the Smatch metric, but lack a well-defined representation and evaluation. |
| Approach: | They propose an algorithm for deriving a unified graph representation using a super-sentential annotation method. |
| Outcome: | The proposed algorithm avoids the pitfalls of over-merging and lacks coherence from under merging. |
RMoA: Optimizing Mixture-of-Agents through Diversity Maximization and Residual Compensation (2025.findings-acl)
Copied to clipboard
| Challenge: | Multi-agent systems based on large language models are limited by high computational overhead, information loss, and robustness. |
| Approach: | They propose a Residual Mixture-of-Agents (RMoA) that integrates residual connections to optimize efficiency and reliability. |
| Outcome: | The proposed model achieves state-of-the-art performance on benchmarks of alignment, mathematical reasoning, code generation, and multitasking understanding, while significantly reducing computational overhead. |
StereoRel: Relational Triple Extraction from a Stereoscopic Perspective (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for relational triple extraction still face challenges, including information loss and error propagation. |
| Approach: | They propose a model which maps relational triples to a three-dimensional space and leverages three decoders to extract them. |
| Outcome: | The proposed model outperforms the baselines on five public datasets. |
CQIL: Inference Latency Optimization with Concurrent Computation of Quasi-Independent Layers (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve inference efficiency target to reduce per-layer latency, but ignore cumulative latency due to number of layers. |
| Approach: | They propose to identify quasi-independent layers that can be concurrently computed to significantly decrease inference latency. |
| Outcome: | Empirical results show that the proposed method reduces latency by 48.3% on LLaMA-33B while maintaining close level of performance. |
OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory (2026.acl-long)
Copied to clipboard
| Challenge: | Existing LLMs are limited by text-context budgets, resulting in token-expensive storage of raw trajectories . Optical Context Retrieval Memory (OCR-Memory) renders historical tra-jectorios into images annotated with unique visual identifiers. |
| Approach: | They propose a framework that leverages the visual modality as a high-density representation of agent experience. |
| Outcome: | Optical Context Retrieval Memory (OCRM) renders historical trajectories into images annotated with unique visual identifiers. |
An Inner Table Retriever for Robust Table Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Table Question Answering (TableQA) is a task of answering NL user questions using factoid answers extracted from table content. |
| Approach: | They propose a method for handling long tables in TableQA that extracts sub-tables to preserve the most relevant information for a question. |
| Outcome: | The proposed method can improve TableQA's accuracy with up to 1.3-4.8% and achieve state-of-the-art in two benchmarks. |
Confront Insider Threat: Precise Anomaly Detection in Behavior Logs Based on LLM Fine-Tuning (2025.coling-main)
Copied to clipboard
| Challenge: | Current methods for insider threat detection suffer from low precision and information loss . a novel approach to detect insider threats is needed to improve accuracy . |
| Approach: | They propose a precise anomaly detection solution based on Large Language Model (LLM) fine-tuning . they represent user behavior in natural language and implement a threat tracing mechanism . |
| Outcome: | The proposed solution achieves an F1 score of 0.8941 on the CERT v6.2 dataset . |
ChuLo: Chunk-Level Key Information Representation for Long Document Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional approaches to truncate inputs, sparse self-attention, and chunking often lead to information loss and hinder the model’s ability to capture long-range dependencies. |
| Approach: | They propose a novel chunk representation method that uses unsupervised keyphrase extraction to group input tokens to retain core document content while reducing input length. |
| Outcome: | The proposed method minimizes information loss and improves the efficiency of Transformer-based models. |
Lost in Embeddings: Information Loss in Vision–Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Experiments reveal connectors substantially distort the local geometry of visual representations, with k-nearest neighbors diverging by 40–60% post-projection, correlating with degradation in retrieval performance. |
| Approach: | They propose two approaches to examine and quantify information loss by analyzing latent representation space. |
| Outcome: | The proposed model improves retrieval performance by analyzing changes in k-nearest neighbor relationships between image representations before and after projection. |
TALON: A Multi-Agent Framework for Long-Table Exploration and Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to query-relevant content retrieval fail to retrieve contextually relevant data. |
| Approach: | They propose a multi-agent framework for table question answering over long tables . TALON features a planning agent that iteratively invokes a tool agent to access tabular data . |
| Outcome: | The proposed framework achieves average accuracy improvements of 7.5% and 12.0% across all language models. |